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ABSTRACT 


A study of several existing longitudinal flying qualities evaluation 
criteria applicable to highly augmented aircraft was psrfoimed. The criteria 
evaluated were: Calspan Neal-Smith; Onstott (Northrop) Time Domain; McDonnell- 

Douglas Equivalent System Approach; R. H. Smith Criterion. Each criterion 
was applied to the same set of longitudinal approach and landing flying 
qualities data. A revised version of the Neal-Smith criterion which is 
applicable to the landing task was developed and tested against other landing 
flying qualities data. Results indicated that both the revised Neal-Smith 
criterion and the Equivalent System Approach are good discriminators of 
pitch landing flying qualities; Neal-Smith has particular merit as a design 
guide, while the Equivalent System Approach is well suited for development 
of appropriate military specification requirements applicable to highly 
augmented aircraft. 
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Section 1 


INTRODUCTION AND PURPOSE 

Pull-euthorit/ electronic augnentetion systeas ere an iaportant 
feature of the latest aircraft designs; witness the F-16, YF-17, F-18A aiixraft 
and the Space Shuttle , Incorporation of the necessary system redundancy 
coupled with the dramatic increase in reliability of modem electronic 
systems has made this "breakthrough" in flight control system design possible. 
Those latest aircraft designs include sophisticated digital flight control 
concepts and, xn two cases, revolutionary pure "fly-by-wire" flight control 
systems. The flight control designer now literally has the capability to 
tailor the aircraft's flying qualities as desired. 

Unfortunately, the potential of this new design power has not been 
realized. In every aircraft design incorporating a sophisticated modem flight 
control system, significant flying qualities problems were evident during the 
final evaluation process. A major factor in creating this undesirable situation 
was the lack of suitable flight control design criteria, or flying qualities 
requirements, applicable to modem highly augmented aircraft. Specifically, 
there is an urgent need for suitable criteria to cover the critical landing 
phase (approach, flare and touchdown). The primary purpose of the study 
presented in this report was, therefore, to explore the available criteri' 
and using a suitable flying qualities data base, attempt to develop approprf ite 
pitch landing flying qualities criteria for augmented aircraft. 

A brief review of the background to this study is in order. The 
demand for increased capability and expanded flight envelopes, in combination 
with the advent of reliable full-authority electronic augmentacion systems, 
have Isd to the evolution of increasingly complex flight control systems. It 
is apparent that in part at least, this additional Cfxaplexity is related to a 
natural desire to implement our new technology rather than to any real design 
requirement. In any event, this additional complexity which is e. ..uent in 
modem designs, although not necessarily a problem in itself, introduces 
significant additional control system dynamics which can potentially alter 
the flying qualities of the aircraft dramatically. For modem, highly 
augmented aircraft such as the F-16, YF-17, F-18A and Space Shuttle 
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(References 1, 2, 3), the response to pilot inputs is "higher order" and cannot 
be adequately described, directly, by the classic aircraft response parameters 
such as those used in MIL-F-8785B (Reference 4) . New flying qualities criteria, 
or requirements, were clearly required. 

In partial response to this need, new longitudinal flying qualities 
criteria were developed, initially directed at up-and-away tracking (Flight 
Phase Category A). Examples are: Calspan Neal-Smith criterion (Reference 5), 

McDonnell-Douglas (MCAIR) equivalent system approach (Reference 6), and Onstott 
(Northrop) time domain criterion (Reference 7). Following these initial efforts. 
Chalk (Reference 8) attempted to extend the Calspan criterion to include the 
landing task (Flight Phase Category C). This extrapolation was based on the 
prevailiiig assvunption that the landing approach pitch task was significantly 
less demanding than the pitch tracking task. Observations made during the 
in-flight simulation phase of the YF-17 development process (Reference 1) 
suggested otherwise. Subsequent in-flight flying qualities research programs 
at Calspan (Reference 9) and NASA Dryden (Reference 10) clearly demonstrated 
that the touchdown phase of the landing task (the last 50 ft) is indeed a very 
demanding pitch task which is comparable to the fighter tracking task. Further, 
the reports showed that the pitch flying qualities of highly augmented aircraft 
can degrade in an explosive fashion (a "flying qualities cliff exists) during 
•ue landing task when significant higher order control system effects (lags, 
or equivalent time delays) are present. The landing task is therefore a very 
critical task and flying qualities criteria are urgently needed which can be 
used to expose deficiencies early in the development process. Recent dramatic 
flight experiences during landing with the Space Shuttle on Flight 5 (Reference 
11), where unexpected and potentially dangerous pilot induced oscillations 
occurred during landing, serve to illustrate this last point. 

In summary, this report presents the results of a study program 
whose purpose was: 

• To review existing longitudinal flying qualities criteria 
applicable to highly augmented aircraft and assess the 
application of the criteria to the landing task. 


• If possible, to develop new landing fl/ing qualities 
criteria for augmented aircraft, 

e To recoomend new research programs or criteria development 
strategies to produce the requisite criteria. 

The primary data base fcr this study is the LAHOS (Landing Approach 
Higher Order System) program. This program (Reference 9) was an in-flight 
investigation using the AFWAL/Calspan NT-33 aircraft to evaluate the longi- 
tudinal flying qualities of a wide variety of representative higher order 
systems. More details on this experiment are presented in Section 2 . 

The remainder of this report is organized as follows: 

e Section 2: Neal-Smith Criterion 

- The results of applying the Neal-Smith 
criterion to the LAHOS data base are presented 
including the development and application of a 
modified criterion. Other data, including the 
Space Shuttle, are also evaluated using the 
modified criterion. 

t Section 3: Onstott (Northrop) Criterion 

- The criterion is reviewed and applied to the 
LAHOS data base; comparisons are made between 
the Neal-Smith and Onstott results and 
modifications suggested. 


• Section 4 


• Section 5 


e Section 6 


t Section ' 


MCAIR Equivalent System Approach 
The application of this method of evaluating the 
landing flying qualities of highly augmented air- 
craft to the LAHOS data base is summarized using 
Reference 12 in which a complete analysis is 
presented. This method is, in fact, the basis 
for the approach specified in the new military 
flying qualities specification, MIL-878SC 
(Reference 13) . 

Ralph Smith Criterion 
The results of applying the criterion 
to the LAHOS data base are presented. 

Study Overview 

Tne results from the study are presented in summary 
form for each criterion examined; overall observa- 
tions are also presented. 

Recommendations 

A brief summary of suggested areas for further work 
to improve existing criteria or develop required 
criteria is presented. 
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Section 2 

NEAL-SMITH CRITERION 

The purpose of this section is primarily to present the results of 
applying the Neal -Smith criterion to the LAHOS data base. A secondary purpose 
is to present a summary of the LAHOS data base to which all the flying qualities 
criteria investigated in this study program are applied. 

2.1 LAHOS DATA BASE (Reference 9) 

The Landing Approach Higher Order Systems (LAHOS) program, to which 
all the criteria in this study program are applied, was an in-flight investi- 
gation of longitudinal approach and landing flying qualities using the AFWAL/ 
Calspan NT-33 variable stability aircraft. 

Briefly, the piloting tasks included realistic instrument and visual 
terminal area tasks, during which actual touchdowns were performed as well as 
intentional approach-only evaluations. Where possible, pilot ratings were 
given for both the flare and touchdown task and the overall terminal area 
task. For the purposes of this study program, the average pilot rating is 
used where multiple evaluations for a configuration occurred. Pilot ratings 
should not, of course, be reviewed separately from the pilot comments which are 
documented fully in Reference 9. 

As shown in Figure 1 and Table 1, the LAHOS experiment variables were 
the short period and the flight control system dynamics. Five short period 
configurations spanning the military specifications were simulated; the phugoid 
and lateral-directional characteristics remained essentially constant. First- 
order lag and lead/lag networks, second-order prefilters, and a fourth-order 
Butterworth filter (approximating a pure time delay) made up the control system 
parameters. In addition, the modified and original control systems of the 
simulated YF-17 (Reference 2) were included. A total of 45 LAHOS configura- 
tions were used in this study program. 

The results from the LA.HOS program clearly showed that the flare and 
touchdown task was the critical piloting area in the overall approach and 
landing task. Severe degradations in longitudinal flying qualities ("cliffs") 
typically occurred in the last 50 ft prior to touchdown for configurations 
with significant additional control system lag. 
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M* - PILOT SELECTED 
®E5 


Figure 1 LAHOS EXPERIMENT VARIABLES 








TABLE 1 

SIMtARY OF LAHOS CONPIGURAnONS 






N'OTES: • Total configuration d>Tiamic model includes feel system and actuator 

dynamics. Time dtlay is approximated using a fourth ordered Butterworth 
filter - described by ^ 3 /^ 3 , (See Reference 9). 

a Configurations 4-0, “-1, and *-3 not used in analysis (see 

Section 2.5) 
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An exception to these observations was evident for the unaugmented, 
lightly damped short period configurations and the unstable cases (Configura- 
tion 7, not used in this analysis). In several instances better pilot ratings 
were given for the flare and touchdown task than for the approach task. In 
fact, no PIO tendencies were observed in the flare and touchdown even with a 
very low short period damping ratio. 

2.2 NEAL-SMITH CRITERION REVIEW 

The Neal-Smith closed loop flying qualities criterion was originally 
developed as a longitudinal flying qualities evaluation tool, or "yardstick", 
for highly augmented fighter aircraft performing precision tracking tasks 
(Flight Phase Category A). An attempt was later made to extend the application 
of the criterion to the approach and landing task (Flight Phase Category C) 
but the results were poor. In this work (Reference 8) the faulty assumption 
was made that the landing task was a low gain, undemanding task relative to a 
fighter tracking task. Subsequent evidence from the YF-17 simulation program 
(Reference 2) and the LAHOS program itself indicated that the flare and touch- 
down phase of the landing task was indeed a demanding, high gain task. 

In the study reported in this section, the Neal-Smith criterion, 
which is based on the applicability of a simple closed-loop pitch attitude 
tracking task, is applied to the LAHOS data from a "fresh" viewpoint. It is 
obvious that the landing task involves more elements than pitch attitude 
control; speed and flight path control are also clearly important elements. 
However, on the basis that good inner-loop attitude control is a necessary, 
but perhaps not sufficient, condition for good approach and landing flying 
qualities, the application of the criterion to the LAHOS data is credible. 

Complete details on the criterion are contained in Reference 5. 
Briefly, the criterion assumes a simple closed-loop pitch attitude tracking 
task as shown in Figure 2. The pilot block in the closed loop should be 
viewed, more properly, as a pitch attitude compensator since even though the 
form of the "pilot model" used is representative, the model was not 
experimentally confirmed. The criterion represents a "flying qualities test" 
and as such is not dependent on the accuracy of the "pilot model" assumed. 
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PITCH AnmjDE COMPENSATOR 


FIGURE 2 CRITERION PITCH TRACKING TASK 



OUTPUTS 


Figure 3 CRITERION PARAMETERS 




The criterion assximes a certain "perfoiiaance standard", or degree 
of aggressiveness, with vdiich the "pilot" closes the loop. This standard is 
defined in the frequency dooain as a bandwidth frequency (Ug) . This bandwidth 
is task dependent; the value for a particular task is dete*iained heuristically 
using pilot rating and comnent data to obtain the best overall correlation with 
the criterion p&raneters. For a given desired bandwidth, the "loop is closed" 
ani the compensator, or pilot model, parameters are varied ta yield the best 
overall closed-loop performance. 

The criterion output parameters are the pilot compensation (workload) 
required and the resulting closed loop performance as measured by the maximum 
value of closed-loop resonance frequency performance is 

constrained by limiting the "droop" up to the bandwidth frequency. These 
criterion parameters are illustrated in Figure 3. 

Evaluation of a specific LAHOS configuration using the Neal-Smith 
criterion consists of the following steps: 

• Specify the bandwidth appropriate for the task; must 
be determined for each task by data correlation (the 
purpose of this study) . 

• Adjust pilot model parameters, the compensation, 

(using a fixed value of time delay) to meet the 
"performance standard" set by the bandwidth 
requirement. 

• Measure the closed-loop compensation required 

(pilot workload) and the closed-loop maximum 

resonance ( i a/ 9 i ). 

c ^ax 

• Typically, pilot workload is measured by the phase 
angle of the compensation required at the bandwidth 
frequency ) . 
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• Plot measured values against Neal-Smlth flying quali,ties 
boundaries to evaluate the flying qualities. Boundaries 
for the original tracking data are shown in Figure 4; 
typical pilot conaents around the Neal -Smith 
parameter plane are illustrated in Figure 5. 

All of this analysis is performed using a digital computer program. 

In the original analysis (Reference 5 ) , a pilot time delay of 
■ 0.3 sec was assumed and a maximum droop of -3 dB was imposed. For the 
flight condition most representative of a fighter tracking and maneuvering 
environment (350 knot case), a bandwidth of 3.5 rad/sec was selected. 

2.3 CORRELATION WITH LAHOS DATA 

As a first step in the process of developing a form of the Neal-Smith 
criterion which is applicable to the landing task, the LAHOS data was reviewed 
using a low bandwidth (1.2 rad/sec) as suggested in Reference 8. Not surpris- 
ingly, the correlation was poor since, as shown in the YF-17 example, the 
landing task is clearly a higher bandwidth task. 

The next step in the correlation process was to use the tracking 
values of bandwidth and pilot delay (3.0 and 3.5 rad/sec and 0.3 sec respectively) 
employed in Reference 5. In this case correlation was better since higher band- 
width is more appropriate for the landing task but significant anomalies were 
still present. 
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LIGHTLY DAMPED 
CLOSED LOOP 
TENDENCY TO PIO, 

OVERSHOOT OR OSCILLATE 

I SLUGGISH 

I INITIAL 



Figure 5 TYPICAL PILOT COhWENTS 
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Finally, It waa dacided to take a "fresh" look at the criterion 
paraaeters (bandwidth and pilot delay) for the analysis of the LAIKIS data. 

As a starting point, attention was centered on twc "benchsurk" configurations 
whose flying qualities were good and were well substantiated. Configurations 
2-1 and 6-2 (modified YF-17) were selected for this purpose. 

The objective was to select values of bandwidth and time delay which 
pxiced the benchmark configurations in sensible locations on the criterion 
plane and further, to provide discrisdnation of the remaining LAHOS data. 

Pilot ratings, connents and discrete error tracking records were used for 
guidance in this correlation process. For reference, the effects of 
increasing bandwidth on the location of a particular configuration on the 
Neal -Smith plane are presented in Figure 6 . 

As shown in the plot presented in Figure 7, the closed-loop 
resonance at a particular bandwidth is significantly affected by the value 
of pilot time delay selected above C.2 sec. Since the "benchmark" configura- 
tions were observed by the pilots to have well damped closed-loop performance, 
it was necessary to select a value of time delay of 0.2 secs to achieve a 
reasonable correlation between the comments and the criterion closed-loop 
performance. 

The criterion parameters for best correlation of the LAHOS data 
which evolved from the correlation process were: 

e Bandwidth of 3.0 rad/sec 

e Pilot time delay of u.2 sec 

All the LAHOS data are presented in Figure 8A on the Neal-Smith 
parameter plane using these criterion parameters and the original flying 
qualities boundaries o'' ; ;ference 5. The LAHOS configuration identifier for 
each data point is presetted on Figure 8B. The grouping of the data is 
comparable to the original Neal-Smith analysis. Configurations with negative 
resonance are a consequence of forcing the criterion low frequency "droop" 
constant. Relaxation of this constraint for configurations with no closed- 
loop resonance concerns is discussed in the next subsection. 
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iqure 6 EFFECTS OF INCREASED BANOHIDTH ON CONFIGURATION 
MAPPING IN NEAL-SMITH PARAMETER PLANE 
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Figure 












Also presented in the next subsection is the final "best" criterion 
for the landing task (Flight Phase Category C) including refined flying quali- 
ties boundaries. A discussion of the anomalies in the final application of the 
revised criterion to the LAHOS data base is presented in Subsection 2.5. 

2.4 REVISED NEAL-SMITO LANDING FLYING QUALITIES CRITERION 

For aircraft configurations with no tendency toward closed- loop 
oscillations (low resonancel, insistance on meeting the original -3 dB droop 
requirement as well as the desired bandwidth can lead to unnecessary additional 
"pilot" cwopensation. To make the compensation more realistic, this droop 
constraint was relaxed for configurations with resonance less than 2 dB as 
shown in Figure 9. 

The final results of the correlation of the LAHOS data on the Neal- 
Smith parameter plane are presented in Figure 10. New slightly modified 
criterion flying qualities boundaires applicable to the approach and landing 
task were required as shown on the figure. 

The correlation is .’’ood; there are, however, anomalies. One such 
group is enclosed in the dashec box. Note that on Figure 10 the configuration 
locations are identical to thost on Figures 8A and B above a resonance of 2 
dB. All the criterion correlation anomalies are discussed in the next 
subsection. 

Since a critical function of a flying qualities evaluation 
criterion is to expose or eliminate systems which have significant problems, 
it is reasonable to consider a correlation "failure" as follows. If the 
predicted flying qualities level is better than the actual pilot rating level, 
then the criterion has failed. Since the criterion is directed at a set of 
requirements simultaneously, it is a reasonable assumption that should this 
rating comparison occur it is because the criterion is wrong and not due to 
other factors not included in the criterion. For the purposes of this report, 
the converse (predicted worse than actual) is not considered to be a failure; 
conservative designs can result in this instance but, unless this situation 
occurs often, it is not a major criterion concern. 


19 








With these correlation rules, the revised Neal-Smith criterion 
correlation is very good, approximately 90%. If all significant incorrect 
predictions are counted, the correlation is approximately 80%. If the 
essentially unaugmented, low damping ratio cases are removed from considera- 
tion the correlation is again ^proximately 90%. 

The test of the revised criterion, which has been "tuned-up" using 
the LAHOS data base, is by application to other landing data. Subsection 2.7 
contains the results of such an application of the criterion to the Space 
Shuttle data and the data from the recent NT-33 Equivalent Systems Verifica- 
tion Program (Reference 14). 

2.5 CRITERION CORRELATICW ANOMALIES 

There are several anomalies in the criterion correlation presented 
in Figure 10. Most bothersome are those cases where the criterion predicts 
better ratings than were given by the pilots, since the prime function of a 
criterion is to provide design guidance which will avoid flying qualities 
problems. The various anomalies are discussed under the following neadings: 

• Low Damping Configurations (3-Series) 

The 3-series configurations without lags (3-C, 3-0, 3-1) were 
predicted by the revised criterion to be worse than observed by the pilots. 

For the configurations with additional lag (3-2 to 3-7), the predicted and 
actual pilot ratings were both Level 3. Although in these cases the resonance 
predicted by the criterion was typically somewhat higher than indicated by 
the pilot comments, the correct flying qualities level was at least predicted. 

The lightly damped cased without lags (3-C, 3-0, 3-1) were rated 
Level 2 but predicted by rhe criterion to be solid Level 3 aircraft. Pilot 
comments suggest that the aircraft were not flown in a closed- loop fashion, 
particularly in the flare and touchdown phase. It is in this phase that the 
pilot flies in a tight closed-loop fashion for the majority of the other 
configurations which have significant initial delay to a pilot input. 


It would appear that for these unaugmented configurations with 
reasonable initial response but oscillator/ final response that the pilot in 
the evaluation environaent is able to fly in on essentially open-loop fashion. 
For example. Pilot A commented that there was "No PIO, just an airplane 
bounce." Poor initial response brought about by an initial delay or lag 
appears to force the pilot to fly in a closed-loop fashion, witness the 
reasonable correlation of the majority of the data. It is entirely possible 
that evaluation of these configurations in moderate to heavy turbulence 
would force the pilot to fly in a closed-loop fashion and result in pilot 
ratings more consistent with the criterion. 

Unaugmented, lightly damped aircraft are not typical of today's 
highly augmented vehicles and are, therefore, not a primary factor in 
evaluating the usefulness of the criterion. However, the fact remains that 
the landing closed- loop criterion does not handle such aircraft. The pilot 
ratings for the low damping configurations without lags are also not consistent 
with the existing requirements (-8785B); again evaluation in low turbulence 
levels may be responsible for these discrepancies. 

• Configurations in the Dashed Box (Figure 10) 

To assist in this discussion, the configurations in question are 
presented in Figure 11. 

Configurations 4-6, 1-B, 4-7 and 1-C all are rated marginal Level 1 
aircraft; when the safety pilot ratings are used for guidance, the placement 
of these configurations on the Neal -Smith plane appears reasonable. For each 
evaluation, the safct) pilot gave an independent rating for the configuration 
based on his observation of the performance. This rating does not, obviously, 
account for ai’y deficiencies related to pilot workload but can be used by the 
analyst to uncover inconsistent performance related ratings. 

Configuration 5-5 is rated Level 3 CPR = 7) but the safety pilot 
rating was significantly different (PR » 4). Since both ratings were based on 
observed performance vrfiich was degraded by a tendency to overcontrol, this 
rating is not considered to be a significant anomaly. 
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Figure 11 LAHOS CORRELATION ANOMALIES 
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The rmtings for ConflgurtCion 4-3 which was evaluated three tines 
show considerable variability. The average rating is around the Level 2 
boundaiy. This configuration appears to be sensitive to pilot technique 
and turbulence conditions. 

Configuration 4-11 was only evaluated once and was given a clear 
Level 3 rating by the pilot; it is, therefore, an anomaly which stands out. 

Configurations 5-5, 4-3 and 4-11 have one feature in comaon: in 

each case, there is a sharp degradation in flying qualities as the task 
lequirement and corresponding pilot "gain", or standard of performance, 
increases near the ground. Note the rating change between approach and touch- 
down shown on Figure 11. The sensitivity of those configurations to changes 
in criterion parameters is reviewed in the next subsection. It appears that, 
for reasons which are not immediately obvious from the comments, the pilot is 
unable to achieve a consistent performance standard for these configurations. 

• Other Configurations of Interest 

Configuration 4-0 from the original LAHOS data set was not used in 
this study. The pilot rating (PR ■ 6) appears inconsistent when compared with 
4-1 (PR 2) which was used as the base configuration for the highly damped 
series. Configuration 4-0 would plot in the Level 1 region but was rated 
Level 2. This inconsistency is mentioned since the trend in modem control 
system designs is toward overdamped systems. A more thorough evaluation of 
very highly damped configurations is required. 

Configuration 1-A appears (see F' ure 10) with a Level 2 s>Tnbol in 
the revised criterion Level 1 region. The configuration was rated Level 1 by 
the Safety Pilot and Level 2 (PR ■ 6) by the pilot; it does not represent, 
therefore, a serious violation. 

2.6 CONFIGURATION SENSITIVITIES TO CRITERION PARAFETERS 

It is clear that some aircraft dj'namic combinations are particularly 
sensitive to changes in task environment or piloting technique. In this 
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context, sensitive means that large changes in flying qualitias can occur 
with different pilots or with small changes in the task standard of 
performance. For these aircraft, large variations in pilot ratings for the 
sane task are common. Indeed, the measure of a good aircraft is its insensi- 
tivity to pilot techniques ox small task variations. From a flying qualities 
requirement viewpoint, application of the criterion at a specific bandwidth 
is likely required; however, from a design criterion viewpoint, evaluation of 
the changes in performance over a realistic range of bandwidths provides the 
more important information. This point is illustrated in the examples which 
follow. 

In the context of the Neal -Smith criterion, the "sensitivity" of a 
configuration can be evaluated by observing the changes in closed-loop per- 
formance (resonance) and pilot workload (compensation) for changes in the cri- 
terion parameters, such as bandwidth frequency. As noted in the last subsec- 
tion, the key smomalous configurations (S-5, 4-3 and 4-11) all have large changes 
in ratings between the approach and touchdown tasks. It is these configura- 
tions which are of particular interest in this subsection. The sensitivity 
of a configuration to changes in criterion parameters can be nicely illustrated 
using carpet plots which show the variation in resonance, vari- 

ous combinations of bandwidth (u>g) , "droop" and pilot lead compensation 
For these plots, pilot time delay is fixed at 0.2 Secs. Lead time constant 
is used for these plots not phase angle of the compensation at the bandwidth 
frequency to clarify the trends. Indeed, it is not entirely clear 

which parameter is the appropriate way to reflect workload. 

e "Good" Configuration (2-1) ; 

The carpet plot for Configuration 2-1 is presented in Figure 12. It 
is clear from the figure why this aircraft was chosen as a "benchmark" config- 
uration (PR “ 2) : at a given value of the droop constraint (say -3 dB, the 

original criterion value), large increases in bandwidth (2.5 to 3.5 rad/sec) 
can be achieved by modest increases in pilot load with very little increase in 
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resonanc*. Th« pilot can fly a variaty of tasks (bandwidths) with this aircraft 
and not sea any draatatic changes in task parforaanca. Ha can respond to that 
unexpected gust or correct an inadvertent error quickly without an explosive change 
in the aircraft flying qualities. 

e "Bad” Configuration (2-9); 

The contrast is evident in Figure 13 which shews the carpet plot for 
this very poor, PIO prone, aircraft (PR ■ 10). For this case, dramatic 
increases in resonance occur with changes in bandwidth . Clearly, this is a 
very ser litive aircraft with a lurking "flying qualities cliff." For the 
approach task, it's not great (PR - S) but the job can be done; in the flare 
with the higher gain task, explosive changes in flying qualities can occur. 


If a flying qualities criterion serves no other inirpose, it must ex- 
pose these configurations. 

e "Marginal" Configurations (5-5, 4-3 and 4-11): 

The carpet plots for these configurations are presented in Figures 
14, 15, and 16. In each case, there is a steep slope of resonance versus pilot 
lead for a given droop constraint. The large changes in pilot ratings be- 
tween tho approach and touchdown tasks are further evidence of this sensitivity. 
These are configurations, therefore, which are impossible to evaluate properly 
by application of the criterion at only one bandwidth. 


From a design point of view, each configuration should be tested 
using a means of observing t** configuration "sensitivity", such as by using a 
carpet plot, before a final flying qualities prediction can be made. 
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There is, therefore, another dimension to the criterion plane; suit- 
able sensitivity parameters are required. From the pilot point of view, this 
sensitivity reflects the degree of difficulty he has in "adapting" (compensating) 
as the task requirements change rapidly. 

e Potential "Adaptability" Criteria; 

Possible "adaptability" metrics are presented in Figure 17. Further 
study is required to determine the exact nature of a suitable adaptability 
criterion. It is entirely possible that the complete criterion can be expressed 
in terms of an adaptability criterion without using a specific bandwidth but 
requiring certain gradients of, say, resonance with pilot lead, over a range 
of bandwidths. 

2.7 APPLICATIONS OF THE REVISED CRITERION 

The first question that arises when a revision is introduced to a cri- 
terion is what the effects of this revision are on the original work. It is 
appropriate, therefore, to first apply the revised criterion to the original 
Neal-Snith data base (Reference 5) . 

• Original Neal-Smith Data: 

For this comparison, the 350 knot data from the original Neal-Smith 
data was selected since this flight condition was observed to be the most 
realistic fighter tracking environment. 

The data are presented in Figure 18 on the Neal-Smith plane using 
the original criterion parameters of = 3.5 rad/sec and t = 0.3 sec and the 
original flying qualities boundaries. 
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LEVEL ONE; PR<3.5 

LEVEL TWO; 3.5<PR<6.5 

LEVEL THREE: PR >6. 5 







Sinco tha bandwidth required is a function of the task, which in this 
case is a fighter pitch tracking task, the original data was evaluated at a 
variety of bandwidths with the pilot delay, of 0.2 sec as used in 
the revised Neal~Sinith criterion based on the LAHOS data* The best correlation 
of the data is presented in Figure 19; criterion parameters were « 4.5 rad/sec 

and T -0.2 sec. Correlation of the data is as good as in the original study. 

p 

Both the revised (see Figure 10) and original boundaries (see Figure 8A) are 
shown; these changes do not appear to be significant. The revised boundaries 
are, therefore, generally applicable. 

It is not surprising that the standard of performance (bandwidth) 
is higher for the fighter tracking task (4.5 rad/sec compared to 3.0 rad/sec). 

The surprising result in the recent studies of flying qualities in the approach 
and landing task is that the landing phase of the task is significantly more 
demanding than previously recognized. In this context, it is worth noting that 
the majority of the data used to develop the Flight Phase Category C data in 
MIL-8785B, and -8785C for that matter, is applicable to the approach phase 
only and not the landing task. For the specifications. Flight Phase Category A 
requirements are more applicable to the real Category C tasks than are the 
present landing approach requirements. 

A review of the Category C requirements for all aircraft classes is 
in order therefore, without regard to the degree of augmentation present. 

• Space Shuttle Data (P.eference 15) 

The Space Shuttle is obviously a unique vehicle; of particular 
interest in the context of this study are the landing flying qualities. 

During numerous studies, simulations, and indeed during the last flight 
itself, the aircraft has exhibited less than satisfactory pitch landing flying 
qualities (see References 11 and 15, for example). In fact, during the last 
flight (Number 5) a pitch PIQ was encountered near touchdown. While there 
are clearly other factors about the design of this unusual craft that influ- 
ence the pitch flying qualities, the equivalent time delay associated with 
the complex digital flight control system is potentially a major source of the 
problem. 
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Figure 19 NEAL-SMITH DATA: 350 KNOT 0 

(ORIGINAL DROOP CONSTRAINT) 








Using the Shuttle model developed for the analysis in Reference 15, 
the Shuttle configuration was evaluated using the revised Neal-Smith Criterion 
for pitch landing flying qualities. The results are shown in Figure 20 along 
with the Equivalent System Program results discussed in the next subsection. 

Also shown on Figure 20 is the large increase in resonance which occurs for a 
modest bandwidth and lead compensation change. 

The criterion confirms what was observed in flight: the Shuttle, 

as represented by the model used in Reference 15, is PIO prone and a marginally 
acceptable vehicle in the landing task. Further, the ’’sensitivity" reflected 
by the relatively large change in resonance with changes in bandwidth indicates 
that the flying qualities are subject to rapid deterioration (a "cliff") with 
changes in task performance standard or pilot technique. In summary, the 
criterion "predicts" the Shuttle characteristics satisfactorily and is therefore 
a useful design evaluation tool. 

Also shown on Figure 20, is the effect of including a "PIO suppressor”, 
which is, in effect, a command gain changer which operates as a function of pilot 
input size and frequency. Background to this analysis, which was part of this 
study, is fully documented in Appendix A. The analysis is approximate since 
a linear description of the non-linear suppressor was necessary for use with 
the frequency-domain Neal-Smith criterion. A suitable time-domain closed-loop 
criterion would not require this approximation. The results of the admittedly 
approximate analysis show that the suppressor does reduce the PIO tendency but 
does not cure the explosive nature of the configuration. 

Although the specific Shuttle "PIO suppressors" were studied in 
Reference 3, more general studies are clearly in order. These studies must 
include simulations with very realistic, stressed, tasks since small changes 
in task or pilot technique are known to have a dramatic t:.fect on the flying 
qualities of PIO prone aircraft. 
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EQUIVALENT SYSTEMS PROGRAM AND SPACE SHUTTLE DATA: 
Up = 3.0 r/s, Tp = 0.2 (Revised Boundaries) 





• Equivalent Systems Verification Data (Reference 14) 

An in-flight simulation program was recently conducted using the 
AFWAL/Calspan NT-33 aircraft to verify the equivalent system approach and to 
evaluate pitch landing flying qualities of highly augmented aircraft. The 
results from the longitudinal portion of this program represents an ideal data 
package since a wide range of high order systems were evaluated in a realistic 
landing task. Accordingly, the major configurations from Reference 14 were 
evaluated using the revised Neal-Smith criterion; the results are presented 
in Figure 20. 

The discrimination of the data is quite reasonable, particularly 
when the preliminary nature of the equivalent system program is considered. 

It was a small program conducted on a compressed schedule and therefore the 
variability of the pilot rating data might be higher than normal. 

As discussed in Section 2.4, for comparison purposes, it is reason- 
able to consider that a correlation "failure" occurs when the predicted flying 
qualities level is better than the actual pilot rating level. In this context, 
the converse (predicted worse than actual) is not a failure, but could possibly 
result in a somewhat conservative flight control system. 

With these correlation rules the criterion correlation is very good, 
approximately 95%. If all incorrect predictions are counted, the correlation 
is 75%, although few of the violations are really significant. 

It should be noted that the same area which produced correlation 
anomalies with the LAHOS data: 2 dB resonance and 50 to 65 deg of phase, also 

has some anomalies in this data set. The anomalies in this case are, however, 
not serious, but would suggest an extension of the Level I boundary; however, 
considering all the data reviewed the revised boundaries shown seem reasonable. 
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2.8 


SUMMARY 


From this study of the application of the Neal-Saith closed-loop 
flying qualities criterion to the landing flying qualities of highly augnented 
aircraft, the following observations can be made: 

• Control of pitch attitude in the landing task is a 
critical, relatively high bandwidth task. 

e A revised version of the Neal -Smith criterion 
proved to be a good discriminator of pitch 
landing flying qualities. For the revised 
criterion: 

- Bandwidth of 3.0 rad/sec and time delay of 
0.2 sec were selected. 

Level 1 and Level 2 boundaries were slightly 
altered. 

Droop requirements were relaxed for non- 
oscillatory aircraft. 

e Bare airframe, lightly damped configurations were not 
adequately evaluated during the LAHOS program and do 
not correlate with the revised criterion. More study 
is required in this area, although such configurations 
are largely of academic interest in the context of 
today's control system designs. 

• More data is required for very’ heavily damped aircraft 
to address the applicability of the criterion in this 
area. 

• The required a priori knowledge of the performance 
(bandwidth) requirements for a particular task, may be 
eliminated by development of a suitable general adapta- 
bility requirement for augment eo aircraft. 
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Section 3 
ONSTOTT CRITERION 

A longitudinal flying qualities criterion developed by Onstott of 
the Northrop Corporation was included in the current study because of the 
criterion's ability to correlate the original Neal-Smith data evidenced in 
Reference 7. Furthemore, because of its time domain formulation, the 
pot' ttial to address non-linear flight control systems is inherent in the 
method. 

3.1 MECHANICS OF CRITERION 

The basis of the criterion is a compensatory pilot model of a step 
target tracking task over a specified time period (Figure 21). The pilot 
model adopts two control modes during the tracking. In the initial acquisition 
phase the pilot model employs proportional and rate attitude compensation. At 
some time, D, switching to a tracking mode takes place and integral attitude 
compensation is added. This mode switching is intended to provide both 
rapid initial target acquisition and precise final tracking. 

Application of the criterion requires; first, maximizing the time 
on target (TOT), defined as the total time for which the pitch attitude is 
within one pipper diameter of the commanded attitude; and, second, the 
calculation of the RMS tracking error. Since TOT is a function of the five 

pilot model parameters {K , T , K , T , K\ ) and the switching time D, 

Pj t-r P|- i-r ^ 

maximizing TOT is a formidable Sptimization problem. 

In Reference 7, pilot ratings for the Neal-Smith data are shown to 

correlate with the two tracking performance parameters, TOT and RMS error. 

The data support the intuitive notion that high TOT and low RMS tracking error 
are indicators of satisfactory flying qualities. It is worthwhile to note that 
this criterion is to some extent, equivalent to the Neal-Smith criterion both 
from the standpoint of the "pilot" model form and the fact that a performance 
standard is employed to determine the model parameters. When correctly derived 
the time domain characteristics, TOT and RMS tracking error, are, in some sense, 
equivalent to bandwidth and resonance in the frequency domain. The most note- 
worthy difference in the two criteria is the lack of a workload metric in the 
Onstott criterion. 
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In the Neal-Smlth Criterion, the level of flyinz qualities is a 
function of both closed loop performance (bandwidth and resonance) and pilot 
workload (phase lead crapensation). In Onstott's criterion, flying qualities 
are not related to the ncgnltudos of the c<»puted pilot compensation. At the 
current level of experience with the Onstott criterion, it is not known 
whether this absence of a workload metric is a fundamental limitation of 
the method. 

The results of applying the Onstott criterion to selected LAHOS data 
and the original Neal-Saith data are presented in the next section. This data 
was generated by Onstott using a recently developed digital computer version 
of his method. During this process a discrepancy was discovered in the 
implementation of the closed-loop method; the impact of this discrepancy is 
discussed in Section 3.3. 

3.2 APPLICATION TO LAHOS AND NEAL-SMITH DATA 

The calculation of the Onstott pilot model parameters is a complex 
optimization problem. In Reference 7 the maximization of TOT was performed 
by manually perturbing each of the parameters and searching for a global 
maximum TOT. It is noted that the rating data from Reference 5 presented in 
Reference 7 are in error in that, for repeated evaluations, the best rating 
was selected as representative of the group. For certain configurations, 
pilot ratings, which the authors of Reference 5 considered anomalous because 
of "learning curve" effects, wert employed. With these errors corrected, the 
correlation of pilot rating with TOT and RMS error required slight shifting of 
the flying qualities boundaries to accommodate the rating changes. Subsequent 
to the initiation of the current study, Onstott developed an automated program 
for the optimization of the pilot model parameters and the computation of TOT 
and R.^’B error statistics. This program was applied first to an abbreviated 
LAHOS data base (Reference 9 with second ordered dynamics excluded) anc. thin 
to recomputing the original Neal Smith data. These new data are plotted in 
Figures 22A and 23A. The configuration identifiers for each data set are 
presented in Figures 22B and 23B. At this point, the reader should be 
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Figure 22B NEAL-SMITH DATA vs TIME HISTORY CRITERION 


44 


RMS ERROR 







RMS ERROR 



forewarned that an implemention error was found in the criterion which biased 
the data and consequently, its correlation to TOT and RMS error. Details 
and ramifications of this error are presented in Section 3.3. 

As can be seen from these plots, the criterion tends to group the 
data by level of flying qualities with low RMS tracking erior and high time 
on target corresponding to satisfactory flying qualities. The converse is also 
true. However, one anomalous trend immediately apparent is the tendency for a 
group of configurations from each experiment idiich received acceptable but not 
satisfactory ratings C3.S<PR^6.5) to fall in the region of low RMS tracking 
error and high TOT. Intuitively this trend appears contradictory to the 
philosophical basis of a performance only flying qualities criterion. 

As a consequence of this anomoly, a more detailed time domain examina 
tion of the closed loop characteristics predicted by the Onstott criterion was 
conducted to determine whether the step response time histories exhibited the 
properties cited in pilot commentary and ratings. It was found that for some 
configurations the analytical responses were substantiated by pilot commerts 
while in others the responses were decidedly incorrect. For example, Configura 
tion 5D from the Neal-Smith data (Reference 5) was described by the pilots as 
PIO prone and received a pilot rating of 8.5 with a PIO rating of 4. Figure 24 
illustrates the closed loop attitude responses predicted both by the Onstott 
and Neal-Smith criteria. It can be seen that both criteria predict responses 
that are lightly damped and oscillatory with about the same frequency. 
Furthermore, the Onstott criterion predicts poor performance (low TOT and 
high RMS tracking error) for this configuration (See Figure 22). For thi= 
case, both criteria predict equivalent closed loop characteristics. 

In another case, 6F from Reference 5, with similar closed loop 
flying qualities deficiencies, different characteristics are predicted by the 
two criteria (Figure 25). The Onstott criterion yields a heavily damped 
first-order like response with slow rise time. Neal-Smith on the other hand 
produces an almost zero damped highly oscillatory response which is consistent 
with the pilot rating of 9 and the PIO rating of 4. In terms of the Onstott 


47 







perfoTTBance parameters, this configuration yields a high RMS tracking error 
and low TOT (Figure 22 ). However, these poor perfoimance characteristics are 
attributable to the sluggish and nearly dead beat response rather than oscilla- 
tions as would be expected. Clearly, for this configuration, the Neal-Smith 
criterion produces time domain results which duplicate the characteristics 
cited by the pilots while the Onstott criterion predicts response at 
variance with these observations. 

3.3 CRITERION MODIFICATION 


As a result of these observations, the program listing for the 
automatic computation of the Onstott performance criteria was examined to 
determine possible modifications which might resolve these anomalies with 
certain configurations. A discrepancy was found with respect to the functional 
block diagram in Reference 7 and the actual computer implementation. The pilot 
model is illustrated as a forward loop compensator which operates on the 
attitude error signal. For example, in the tracking mode, the command to the 
aircraft is given by: 


« =fdelau T J < K ( 0E (t) + T. 0E (t) + K 
F { Pf V 



However, the error rate term 0^ is implemented as 0^ = - 6 

rather than 0, - 0^ , which implies that only the proportional and integral 
terms are series compensation while the rate term is feedback compensation 
only. 

The effect of this implementation on the tracking mode is illustrated 
in the block diagram of Figure 26. 
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Figure 26 CLOSED-LOOP BLOCK DIAGRAM AS IMPLEMENTED 

The acquisition mode is similarly affected. The block diagram for 
this mode can be realized by setting Kj 0 and redefining the nomenclature 
of the command and feedback gains. ^ 

As a check, the calculation of 0^ in the time history subroutine 
of the Onstott criterion program was corrected and selected cases were rerun. 

In each of these cases, the correction had the desired effect, in that, 
configurations with erroneous damped responses now tended to take on oscillatory 
characteristics while configurations which were substantially in agreement both 
with pilot comments and ratin'js and the Neal-Smith responses were little changed. 
For example. Figure 27 indicates that Configuration 6F from Reference 5 now 
exhibits a large initial overshoot and bobble. The final portion of the 
response, however, indicates a rapid convergence to the commanded attitude 
which is attributable to the integral of attitude compensation. 

Figure 28 is a comparison of time histories for Configuration 2D from 
Reference 5 using the original and modified Onstott criterion programs and the 
original Neal-Smith program. Note that each response is similar and in agree- 
ment with the pilot rating and PIO rating. For this case, the program 
modification had little effect on the result. 
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In the Onstott criterion parameter plane the effect of the program 
modification is illustrated in Figures 22 and 23. For the Neal-Smith cases 
illustrated in Figure 22, the program modification resulted in only small 
changes in TOT and RMS error for the cases which were initially in good 
agreement with pilot ratings and comments (2D, 5D) while Configuration 6F 
exhibits a large increase in RMS error. For these data, the program modifica- 
tion appears to increase the discrimination of the criterion. 

From the LAHOS data, an anomalous configuration with low RMS error 
and high TOT was examined (Configuration 4-3). As can be seen ixom Figure 23, 
the effect of the program modification is to increase the RMS error considerably 
and reduce the TOT. The resulting position in the parameter plane is more 
appropriate to the unacceptable pilot rating which this configuration received. 

3.4 SUI«4ARY 

The foregoing analyses indicate that the Onstott criterion has merit 
but exhibits anomalous results for certain configurations. Specifically, the 
tendency to place some configurations with Level 2 and 3 pilot ratings in 
the high TOT, low RMS error region of the parameter plane, and occasionally 
predict damped responses for configurations which are in fact oscillatory and 
PIO prone, is not reasonable, particularly for a performance based criterion. 

The tendency to predict erroneous closed loop time response characteristics for 
certain configurations appears to be related, however, to an error in the 
calculation of pilot lead compensation. Examination of selected cases with 
this error corrected produces both time histories and tracking statistics 
that are more in agreement with pilot comments and ratings. The single case 
from the LAHOS experiment (Configuration 4-3) suggests that the anomalous 
group of Level 2 high TOT, low RMS error configurations may move to more 
appropriate regions of the parameter plane with the 0^ computation corrected. 

A major improvement in the application of this criterion would be 
to reduce its computational complexity. One change recommended for 
consideration is to eliminate the dual mode feature and operate with a pro- 
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portional plus rate compensation only. This simplification would eliminate 
the switching time D and the integral gain from the TOT state vector 
(i.e. 6 parameters reduced to 4) and should produce a considerable reduction 
in computing time. 

Another factor for consideration is the current lack of a workload 
measure in the criterion as presently formulated. It seems intuitive that 
configurations with identical performance statistics may receive significantly 
different pilot ratings depending on the pilcc compensation (i.e. workload) 
required. 

It is recommended that further development of a time domain criterion, 
such as the Onstott technique described here, be pursued. The current trend 
in PCS design, toward non-linear and even time varying command and feedback 
networks has created a need for flying qualities design and aitalysis techniques 
capable of addressing these systems without resorting to cumbersome and 
possibly inappropriate approximations such as linearization and describing 
functions (see for example Appendix A). 
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Section 4 


MCAIR EQUIVALENT SYSTEM APPROACH 


The equivalent system approach to the evaluation of the longitudinal 
flying qualities of highly augmented aircraft involves the representation of a 
high order aircraft dynamic system with a system of lower order. If a classical 
aircraft response is chosen then the potential arises that the results are 
compatible with the present requirements in the military specification. In 
this case, the equivalent parameters wouM be used and the only additional 
complexity would involve the method used for deriving the desired equivalent 
system. 

The McDonnell-Douglas Company (MCAIR) has been a leader in the 
evolution of this concept; for example, see Reference 6 which fully describes 
their approach. The purpose of this section is to discuss the application 
of the equivalent system approach to the LAHOS data base. 

4,1 MCAIR EQUIVALENT SYSTEM 

To clarify the discussions, a very brief review of the MCAIR equiva- 
lent system approach is in order. The form of the low order transfer function 
used by MCAIR to match the high order, complex system is; 
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This transfer function is the equivalent, constant speed, short-period 
transfer function with a transport time delay added to provide the ability to 
match the high order system. A digital computer program is used to produce 
the best Bode plot match over a frequency range of typically 0.1 to 10 rad/sec 
using a suitable "cost function" to weight the amplitude and phase errors properly. 

The method produces equivalent values of frequency, damping ratio 
and n^/a (when variation of necessary for a "match") and time delay. 

These values can tlien potentially be used to evaluate the flying qualities of 


S6 


the aircraft by cooqiarlson with the appropriate requireaents in the specifica- 
tion. 

MIL-F-8785C (Reference 13), the proposed revision to the exi?*ing 
military ipecification, incorporates the equivalent system approach but, 
unfortunately, does not specify how the equivalent system should be derived. 

4.2 CORREUTION WITH THE LAHOS DATA 

Fortuitously, the LAHOS data have been thoroughly analyzed by MCAIR 
in Reference 14 using their equivalent system method. The results of this 
analysis are, therefore, used for the following discussion. 

Since the equivalent system transfer function has at least 3 

1 /, 

variables and sometimes a fourth (when Bo is varied), several mil-spec 
requirements are involved in the evaluation of a conriguration' s flying 
qualities : 


For the purposes of this correlation study, the new requirements 
from the proposed new mil-spec, MIL-F-87S5C were used: 


0 u Category A and Category C 

0 Allowable Phase Angle at w Boundaries 

Level 1 15 deg 

Level 2 : 30 deg 

Level 3 60 deg 

• Time Delay Boundaries 

Level 1 : .10 sec 

Level 2 : -20 sec 

Level 3 : .25 sec 


The correlation of the equivalent system data for the uAHOS configura- 


tions Is presented In Table 2. The predicted flying qualities levels based on 
each requirement are presented along with the actual pilot rating level assigned 
in the LAHOS experiment. 


For the evaluation of che correlation results, the following rules 

were used: 

e When the assigned pilot rating Level was worse than the worst 
predicted Level using all the requirements, the prediction 
was judged to be incorrect. Unlike the previous criterion 
evaluations, multiple requirements are necessary in 
conjunction with the equivalent system approach. In fact, 
five separate requirements must be evaluated to test 
compliance. 


Level 

1 : 


PR « 

3.5 

Level 

2 : 

3.5 < 

PR « 

6.5 

Level 

3 ; 

6.5 < 

PR < 

9.0 

Level 

4 : 


PR > 

9.0 


No distinction was made between Levels 3 and 4 for this 
correlation evaluation. 

• As for the previous criterion evaluations, when the assigned 
rating was better than the worst predicted Level for the 
requirements, the prediction was not judged to be a failure. 

On this basis, the correlation results are: 

• Using -8785C with only the new phase angle requirement, 84% 
of the cases were correctly predicted. 

• Using -8785C with only the new time delay requirement, 89% 
of the cases were correctly predicted. 

• Using -8785C with both the new phase angle and time delay 
requirements, the correlation was 91%. 
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• Use of Category A vice C frequency boundaries gave 
slightly better correlation; however* with the new 
-8785C control systen dynamic response requirements 
incorporated, the difference was not significant. 

e The equivalent systta method coii'ectly predicted the 
flying qualities level without using the previous 
correlation restriction for approximately 8S% of the 
cases. 

e Use of time delay boundaries appears to be more 

appropriate than the phase lag form of the control 
system dynamics requirement. 

4.3 SUMMARY 

This review of the applicability of the MCAIR Equivalent System 
Approach as an evaluation criterion for pitch landing flying qualities has 
shown that: 

e The approach is an excellent discriminator of pitch 

landing flying qualities for highly augmented aircraft 
when the new time delay and phase lag requirements 
of the proposed MIL-F-8785C are used. 

e Incorporation of the equivalent syston approach into 

the new military specification (MIL-F-8785C) , therefore, 
has merit. However, much of the progress represented 
by this step may have been lost since the method of deriving 
the equivalent parameters is left an open issue. A well 
established method such as that used by MCAIR should be 
specified. 
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• The equivalent s/stea approach is ideally suited to express 
specification requirenents since to a large extent existing 
requirements can be used. Use of equivalent systems does 
not allow evaluation of the effects of combinations of 

marginal characteristics ("combination of bads"); of course, 
the present and proposed specifications have the same deficiency. 
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Section S 

R. H. SMITH CRITERION 


An open loop longitudinal flying qualities criterion, developed by 
R. H. Smith, was included in the current study because of the success exhibited 
in correlating the results of the Neal-Smith experiment (Reference 16] and in 
analyzing the Shuttle landing PIO incident (summarized in the handout document 
supplementing R. H. Smith's presentation at the NASA Houston Shuttle Landing 
PIO briefing by industry representatives including Calspan). It is recognized 
that this criterion has been evolving over a period of several years and this 
material may not represent the current state of development. For background 
information describing the rationale for the criterion and its evolution. 
Reference 17 and 18 should also be consulted. In this section, only the 
mechanics of applying the criterion to the LAHOS data will be described. 

5.1 MECHANICS OF CRITERION 

The R. H. Smith criterion is comprised of two parts directed to first, 
the short term attitude response dynamics and second, the PIO tendency 
through the mechanism of pilot normal acceleration tracking. In the LAHOS 
experiment, n /a at the approach speeds employed, is so low that it is 
doubtful that a pilot n loop closure is a viable cause of PIO. Accordingly, 
the PIO aspect of the criterion was not included in the current study. The 
pitch attitude criterion is comprised of the following metrics: 


0) i 03 ^ uytLS 
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The first requirwnent dictates that the peak pitch rate following a stop 
command must be within the designated limits for Level 1 pilot rating. It is 
intended to exclude configurations with either sluggish or abrupt initial 
response, /^plied to the Neal-Smith data base, no configuration udiich failed 
this criterion received a satisfactory pilot rating. 

Application of requirements (2) and (3) requires the calculation of a 

criterion frequency, as illustrated in Figure 29. As can be seen from this 

figure, satisfaction of requirement (2) is equivalent to an upper limit on 

i. e. u <5.76 rad/sec. 
o 

Requirement (3) was developed primarily from the Neal-Smith data base. 
When the pilot ratings from that experiment were plotted against ^ 
it was found that all the data were bounded by two close parallel lines which, 
in effect, defined a pilot rating functional. (See Figure 30 ). 

Taking the mean of the functional as representative of the average 
pilot ratings, limits for Levels 1, 2 and 3 flying qualities were established 
as defined in requirement (3) and illustrated by the hatched boundaries in 
Figure 30. 

5.2 APPLICATION TO LAHOS DATA 

The LAHOS data base was evaluated with respect to these requirements. 
All of the LAHOS configurations passed requirement (2), that is ui <5.76 rad/ 

w 

sec. Averaged pilot ratings for each configuration are plotted versus 
1 0 

< p— (j,, ) in Figure 30. Configurations which failed the rate response 

requirement (1) are designated by flags. No configurations receiving Level 1 
pilot ratings failed requirement (1). Unlike the Neal-Smith data base, these 
pilot ratings are not bounded by the pilot rating functional. Table 3 
summarizes the criterion’s overall capability in *:erms of predicted versus 
achieved ratings. It can be seen that the criterion is conservative in that 
in only 4 cases are worse ratings achieved than were predicted. In the sense 
that the achieved ratings were better than or equal to predicted ratings the 
criterion was 91% successful with the LAHOS data. However, because of this 
conservatism, the criterion may tend to lead to overdesign, since 36% of the 
configurations were rated better than predicted. In terms of achieving the 
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ATTITUDE PHASE CRITERION APPLIED TO LAHOS DATA 



TABLE 3 


PREDICTED VERSUS ACHIEVED RATINGS FOR LAHOS DATA 
USING R. H. SMITH CRITERION, FAILURES of 
REQUIREMENT INCLUDED 



Predicted Level 


predicted Level of flying qualities, the ratio is only 56% as would be expected 
from the scatter evidenced in Figure 30. 

Although overall the data exhibits scatter, tracking the variation 
in pilot rating for a given short period configuration as the prefilter 
dynMics are changed reveals a functional independence of pilot rating on 
^ F~ (Figure 31). It appears that for a given short period, the 

0 vS w 

a 

rating data approximates the shape of the pilot rating functional but the 
locus for ea-'h short period configuration is displaced by some phase increment. 
Further examination of this characteristic was not within the scope of the 
current program. However, it is likely that a modification to the equation 
for the criterion frequency could be devised to eliminate this phase 
increment and is justifiable on the basis that the LAHOS data was gathered 
in the context of a different task than the Neal-Smith data. 
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Figure 31 ILLUSTRATION OF EFFECT OF SHORT PERIOD DYNAMICS 
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5.3 


SUMMARY 


In <!uianary, it is concluded that although this criterion is reasonably 
effective in screening out configurations with poor flying qualities, it may 
lead to overdesign because it is a relatively poor discriminator of pilot 
rating. It appears that modifications to certain aspects of the criterion 
such as the definition of w may considerably improve the correlation with the 
LAHOS data. 
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Section 6 
STUDY OVERVIEW 


The purpose of this section is to provide e sumaary of the results 
of this study which was directed at the evaluation of several existing pitch 
flying qualities evaluation criteria for highly augmented aircraft in the 
landing flight phase. Recommendations for further work in support of the 
continued develojmient towards suitable flying criteria for today's aircraft 
with complex flight control systems are presented in Section 7. 

6.1 REVISED NEAL-SMITH CRITERION 

• Desirable Features : 

Good pitch landing flying qualities discriminator; 
exposes bad aircraft consistently. 

• Parameter plane dimensions are directly related to 
typical pilot comments. 

- Provides a design target area which guarantees 

good flying qualities if met regardless of system 
complexity. 

Evaluation of aircraft's longitudinal maneuvering 
response characteristics can be done in one step; 
eliminates "combination of bads" question present 
in other criteria and militar>’ specification. 

Ideal as a design criterion since "sensitivity" of 
the aircraft dynamic system to changes in task 
performance standard or pilot technique can be 
explored effectively. 

• Undesirable Features : 

Application of the criterion is relatively complex 
although it can be done efficiently and consistently 
using the digital computer program. 
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Did not prodict pitch landing flying qualities accurately 
for lightly daaped unaugaented aircraft; n»re data in 
high turbulence is required for a proper evaluation of 
criterion. 

Requires an additional "adapt^ility" metric to 
evaluate properly aircraft which are sensitive to task 
variations or changes in pilot technique. The criterion 
does, however, lend itself to such a development. 

Cannot accurately evaluate systems with non-linear 
elements. 

6.2 ONSTOTT (NORTHROP) CRITERION 

• Desirable Features ; 

Concept of a time-domain closed-loop criterion has 
merit; parameters can be related to a real-world 
piloting task. 

Complex flight control systems with non-linear elements 
can potentially be evaluated directly through exact 
modeling in contrast to frequency domain analyses. 

Good potential for modification into a viable 
criterion. 

e Undesirable Feature.* : 

Does not discriminate either pitch landing or 
tracking flying qualities dat.i in a realistic 
fashion in its present form. 

Not correctly formulated; modifications required 
to introduce consistent "performance standard" 
to concept. 

Required computer tine to perform necessary 
optimisation required excessive computer time 
(or a very fast computer). 
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6.3 


MCAIR EQUIVALENT SYSTEM APPROACH 


• D<slr\bl« Peaturts ; 

O 

Good pitch landing flying qualities discriainator when 
new -878SC control s/stea requireaents are used; 
exposes bad aircraft consistently. 

- Approach is couched in teras of the saae parameters 
as the current and proposed specifications. 

e Undesirable Featv , : 

The method of deriving the equivalent system parameters 
is the subject of nuch d'bate; proposed new military 
specification (-8785C) has left the subject open vdiich 
may negate the usefulness of the inclusion of the 
approach as a method to evaluate highly augmented 
airci’aft. 

3 or 4 separate requirements must be used to evaluate 
the pitch maneuvering response characteristics; 
"combination of bad" qualities case is not, therefore, 
covered . 

Sensitivity of an aircraft's flying qualities to 
changes in task performance standard or pilot 
technique cannot be evaluated using this approach. 


The relationship of equivalent modal parameters to 
physically meaningful aircraft parameters or flight 
-.ontrol characteristics is neither obvious nor 
readily determined. See, for example, LAHC:. 
Configuration 2-10 in Reference 14. 
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6.4 


R. H. SMITH CRITERION 


• Desirable Features ; 

Appears to expose bad aircraft but is generally not a 
sufficiently sensitive discriminator of pitch landing 
flying qualities. 

Relatively simple to apply. 

• Undesirable Features : 

Not sufficiently sensitive; for example. Level 3 
phase angle region contains pilot ratings which 
range from 3 to 10. 

Parameters are somewhat abstract eind do not relate 
directly to piloting task. 

6.5 SUMMARY COMMENTS 

The question of compliance demonstration is an important factor in 
the assessment of the suitability of a particular criterion as a flying 
qualities requirement. All criteria assessed in this study require a 
complete description of the augmented aircraft. Modem flight test techniques 
car be used to obtain the required information; however, the process is some- 
what laborious eind potentially open to a variety of interpretations. Other 
approaches, which rely on metrics measured directly from the augmented aircraft 
time history response, such as in Reference 19, could potentially eliminate 
this problem. 

Of the criteria reviewed in this study, the revised Neal -Smith 
criterion and the MCAI.l Equivalent System Approach are both adequate crit- • o.. 
for evaluation of the pitch landing flying qualities of highly augmented 
aircraft. The Onstott method shows promise but needs extensive modifications, 
Ra_ph Smith's criteiion just is not sensitive enough, in its present form at 
least, to be considered an adequate general criterion. 
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In sunnoary, the revised Neal-Snith criterion appears to be the best 
design guide, while the Equivalent Systrai Approach Is the best method to use 
as a basis for modification of the requirements in the present military 
specification to cover highly augmented aircraft. 






Section 7 
RECOMMENDATIONS 

This study program assessed the applicability of several existing 
pitch flying qualities criteria as pitch landing flying qualities criteria and 
was directed at the development, if feasible, of suitable revised or new 
criteria. As a result of this exposure, the following recommendations are 
presented: 

(1) The revised Neal>Smith criterion should be tested further using 

other sources of pitch landing flying qualities data. 

(2) More pitch landing flying qualities data are required. Of 

particular interest are: 

e Configurations which are "sensitive" to small changes 
in task standard of performance or pilot technique. 

• The effects on flying qualities of realistic digital 
mechanizations and their associated equivalent delays. 

An experiment is required in which a variable digital 
flight control system can be evaluated using realistic 
critical tasks. 

• The effects of additional representative control system 
dynamics on the flying qualities of highly augmented 
aircraft with heavily damped responses. 

• The flying qualities of lightly damped unaugmented 
configurations, although such a study would not be 
pertinent to modem augmented aircraft designs. 

• The effects of command gain on the flying qualities 
of highly augmented aircraft with significant initial 
delay in the response to pilot inputs. 
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(3) Specific attention is required to understand the influence of 
Class on landing flying qualities; a large aircraft landing flying 
qiialities experiment sponsored by the USAF and NASA is currently 
in progress using the AFWAL/Calspan TIPS in-flight simulator which 
should provide data relative to this area. 

(4) Further study is required to understand the nature of "sensitive" 
aircraft - aircraft whose flying qualities degrade rapidly, and 
typically unexpectedly, with changes in pilot techniques or 

task standard of performance. Sensitivity (adaptability) 
metrics should be developed with which such aircraft can be 
properly evaluated. 

(5) Further work is required to develop a closed-loop time-domain 
criterion using the work presented in this study as a starting 
point. Such a criterion is necessary to evaluate non-linear 
flight control system mechanizations. In addition, the Onstott 
criterion should be evaluated using the corrected version of the 
criterion and all the available data. 

(6) The NASA non-linear suppressor shows potential as a method for 
reducing the PIO tendencies of an aircraft. The capabilities 
of the suppressor cannot be fully assessed without very careful 
in-flight evaluations which include realistic, critical tasks 
such as touchdowns and actual refueling "plug-ins." As 
demonstrated in numerous evaluations, the explosive nature of 
the flying qualities of a poor aircraft can be missed with 
small changes in task standar.! of performance or pilot technique. 

(7) Finally, very little is known about the effects of control 
system d;/namics on the world of lateral-directional flying 
qualities for all tasks. Clearly, a substantial data base is 
required and suitable exp<iriments must, therefore, be conducted. 
One such experiment which will contribute to the required data 
base is presently being conducted using the AFWAL/Calspan NT-33 
in-flight simulator. 
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APPENDIX A 


SHUTTLE PIO SUPPRESSOR ANALYSIS 

By 

Norman C. Weingarten 


This appendix documents a brief analysis of the closed loop behavior 
of the NASA PIO Suppressor. The suppressor is highly nonlinear (frequency and 
amplitude dependent) and is not easily analyzed in any of our frequency domain 
closed loop programs. Therefore, an approximation of its mechanization was 
developed. This approximation was essentially a gain schedule as a function 
of frequency without phase lag. The gain is actually a function of the 
amplitude of input, but this functional relation was eliminated by assuming 
that the pilot flew with basically one amplitude versus frequency function. 

The analysis requires an assumed Shuttle PIO Suppressor gain shape: 
estimat&d amplitude ratio versus frequency at various amplitude inputs through 
the PIO suppressor. For the PIO suppressor model, the following assumptions 
are made. 


Assume: 1. No dead zone of 1.15° 

2. Reference gain of .593 (@ low frequency ■* in = 0.2 r/sec) 

( low amplitude ■* 6 . -5) 

3. is a function of frequency of input measured by amplitude 
ratio of following filter: .75 

(s+10)^ 

K = 1-3.846 (ARJ direct ratio (not db) 

4 

K limit -.15 0 = .299 

^min 


A-1 


so that 



pilot stick 

gain into = [.36 -i- K (.04d4)\6. |] 

control 

systent 


depends on frequency (K^) 
depends on amplitude 



Obtain AJi in dB of above gains with respect to reference gain 
at 0.593 (low frequency, low amplitude) 
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,2 
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■h,13 

*2,88 

*3,0? 

*5.03 

*5,26 

.5 

- .56 


*2,05 


*4,0? 


1, 

-1,19 


*1,12 


*2,94 


2. 

-2,85 


-1,59 


- ,48 


2,8 

-4,34 


-4,34 


-4,34 


3. 

-4,68 


-5,04 


-5,41 


3,3 

-5,25 


-6,30 


-?,4? 



These data are plotted in Figure A-1 . 


To obtain one Amplitude Ratio reduction curve for the Shuttle PIO 
Suppressor, the pilot was assumed to fly with low amplitudes (5 deg or less at 

frequencies less than 1. rad/sec) and gradually Increase amplitude at higher 
frequencies in a PIO situation (to about 10 to 15 degrees at 3.3 rad/sec where 
the gain reduction stops). This approximate gain change with frequency would be 
for the above shuttle configuration: 


0 db 
7 db 





U) 


(Also see Figure A-1 for chosen gain schedule). 

To see how this gain change with frequency works, it was demonstrated on a PIO 
prone LAHOS configuration (6-1), with various levels of attenuation, and using 
1. and 3.3 rad/sec as the break points and attenuations of -2, -4, -6, -8 dB as 
the final attenuation values, the was 3.0 rad/sec and pilot delay was .2 
seconds - the criterion parameters for the revised Neal -Smith landing 
criterion. 
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Figure A-1 PIO SUPPRESSOR GAIN AHENUATION SCHEDULE 




Results from Configuration 6-1 


Pilot gain as a function of frequency was run on a digital computer 
general closed loop program. (See results plotted in Figure A>2.) This 
figure illustrates the change in closed loop resonance and droop for given 
pilot lead compensation necessary to satisfy the bandwidth recpiirement as the 
’’suppressor" gain attenuation increases through -9 dB. 


The graph indicates that with the suppressor active; 


1 . 

2 . 

3. 

4. 

5. 

6 . 


l>’'ad 


Pilot gain K increases 
Droop decreases 1 hold! 

Resonance decreases 
'^lead “3 dB droop increases 
Resonance at for -3 dB droop decreases 


constant 


Pilot Gain K at t 


lead 


for -3 dB droop decreases a little 


It appears that the PIO Suppressor working on Configuration 6-1 does reduce 
the resonance and droop at a constant x y but the resonance values remain 
very high. To achieve the -3 dB droop would require more pilot compensation 
(increased ) which would reduce the resonance further. However, this 

may not be what the pilot would try to do, since it would be in the direction 
of increasi: j droop. 


Results 'Jsing Shuttle Configuration 


Running the PIO Suppressor with the 0 to -7 dB schedule on the 
Shuttle transfer function showed similar results. The Shuttle model used for 
this analysis is documented fully in Reference 15. It was run at » 2, 2.5, 
and 3.0 rad/sec with a delay for the pilot a;'.d control input as .30 and .06 
seconds. (See Figure A-3 and A-4 for i.jp *2.5, 3 and total delay ■ .36 sec 
results) . 

This part of the analysis was run with the original Neal-Smith pilot 
time delay of 0.3 sec not the revised landing criterion value of 0.2 sec. 
Howev?r, the additional control delay should hav s been .I."' sec rather than .06 
sec as used in the model in Reference 15 and in Section 2.’. 


These results 











with a total delay of .36 secs instead of .32 secs are therefore not exact but 
are sufficient for the purpose of this approximate analysis. 

Reduced resonance and droop is evident at constant with the 

suppressor. HOwever« at lower bandwidths less reduction in resonance is shown. 
It should be noted that as bandwidth increases, droop decreases while resonance 
increases at constant . 

The Neal-Smith solution points indicate that for a given bandwidth, 
the suppressor reduces the resonance (more at higher bandwidths) but requires 
mure pilot lead to achieve the criterion standards. The relatively large 
variation in resonance with increases in bandwidth (sensitive configuration) 
is still present with the suppressor working. 

SUMMARY 

It appears from this brief and sc-aewhat crude amalysis that the PIO 
suppressor does reduce PIO tendencies (reduced closed loop resonance). The 
"sensitivity" of a poor configuration - change in resonance with changes in 
bandwidth - were not, however, significantly altered. Another point, not 
addressed in this analysis, is the shape of the transient response to pilot 
inputs with the suppressor. For step-like inputs the response resembles a 
first-order response with a large lag. For rapid inputs before the suppressor 
can change the gain, this effect might just make an already poor aircraft 
worse. 

It is clear that very careful simulation is required before the 
capabilities of the PIO suppressor can be confirmed. Such a simulation must 
include very realistic, highly stressed tasks. An in-flight simulation which 
includes touchdowns and an actual in-flight refueling task would present an 
appropriate test. 


